Multi-party holomeetings: toward a new era of low-cost volumetric holographic meetings in virtual reality

© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes,creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.Fueled by advances in multi-party communications, increasingly mature immersive technologies being adopted, and the COVID-19 pandemic, a new wave of social virtual reality (VR) platforms have emerged to support socialization, interaction, and collaboration among multiple remote users who are integrated into shared virtual environments. Social VR aims to increase levels of (co-)presence and interaction quality by overcoming the limitations of 2D windowed representations in traditional multi-party video conferencing tools, although most existing solutions rely on 3D avatars to represent users. This article presents a social VR platform that supports real-time volumetric holographic representations of users that are based on point clouds captured by off-the-shelf RGB-D sensors, and it analyzes the platform’s potential for conducting interactive holomeetings (i.e., holoconferencing scenarios). This work evaluates such a platform’s performance and readiness for conducting meetings with up to four users, and it provides insights into aspects of the user experience when using single-camera and low-cost capture systems in scenarios with both frontal and side viewpoints. Overall, the obtained results confirm the platform’s maturity and the potential of holographic communications for conducting interactive multi-party meetings, even when using low-cost systems and single-camera capture systems in scenarios where users are sitting or have a limited translational movement along the X, Y, and Z axes within the 3D virtual environment (commonly known as 3 Degrees of Freedom plus, 3DoF+).The authors would like to thank the members of the EU H2020 VR-Together consortium for their valuable contributions, especially Marc Martos and Mohamad Hjeij for their support in developing and evaluating tasks. This work has been partially funded by: the EU’s Horizon 2020 program, under agreement nº 762111 (VR-Together project); by ACCIÓ (Generalitat de Catalunya), under agreement COMRDI18-1-0008 (ViVIM project); and by Cisco Research and the Silicon Valley Community Foundation, under the grant Extended Reality Multipoint Control Unit (ID: 1779376). The work by Mario Montagud has been additionally funded by Spain’s Agencia Estatal de Investigación under grant RYC2020-030679-I (AEI / 10.13039/501100011033) and by Fondo Social Europeo. The work of David Rincón was supported by Spain’s Agencia Estatal de Investigación within the Ministerio de Ciencia e Innovación under Project PID2019-108713RB-C51 MCIN/AEI/10.13039/501100011033.Peer ReviewedPostprint (published version

A customizable open-source framework for measuring and equalizing e2e delays in shared video watching

Low-latency and media sync are essential requirements to enable interactive multi-party services, such as Social TV. In this work, we present an open-source and customizable framework that allows measuring end-to-end (e2e) video delays and provides support for different type

Impact of Social Media on TV Content Consumption: New Market Strategies, Scenarios and Trends

The mass adoption of Social Media together with the proliferation and widely usage of multi-connected companion devices have tremendously transformed the TV/video consumption paradigm, opening the door to a new range of possibilities. This Special Issue has aimed at analyzing, from different point of views, the impact of Social Media and social interaction tools on the TV/video consumption area. The targeted topics of this Special Issue and a general overview of the accepted articles are provided in this Guest Editorial

Wersync: A WEB-BASED PLATFORM FOR DISTRIBUTED MEDIA SYNCHRONIZATION AND SOCIAL INTERACTION

This paper presents Wersync, which is an adaptive and accurate web-based platform that enables distributed media synchronization and social interaction across remote users. By using Wersync, users can create or join on-going sessions for concurrently consuming the same media content with other remote users in a synchronized manner. Besides, social interaction is provided by sharing the navigation control commands and by integrating synchronized text chat channels. Additionally, two social presence mechanisms have been added to stimulate the participation of external users in on-going sessions on Wersync. By exclusively relying on standard web-based technologies, this platform can guarantee cross-network, cross-platform and crossdevice support, which is a key point in the current heterogeneous media delivery ecosystem

Review of Media Sync Reference Models: Advances and Open Issues

The advances on multimedia systems have brought new challenges and requirements for media sync. Over the years, many media sync solutions have been devised. Due to this variety, several studies have surveyed the existing solutions and proposed classification schemes or reference models for media sync. This paper claims the relevance of media sync reference models to systematically structure and synthesize this research area. Accordingly, a review of the existing reference models is provided, by examining the involved features, components and layers in each one of them. Likewise, some inconsistencies, open issues and missing components in existing reference models have been identified. Accordingly, this study reflects the need for a new modular and extensible theoretical framework or reference model to efficiently comprehend the overall media sync research area

Más allá de la Realidad Virtual. Reconstrucción Acústica Virtual de antiguos teatros del Siglo de Oro. Hacia una vivencia virtual del teatro en los albores de la ultra-realidad

Nowadays, the virtual reconstruction of old and lost buildings is a hot-topic in cultural heritage. Traditionally, this reconstruction has been made with 3D graphical information, but the feeling of immersion can significantly increase with the addition of other senses. Moreover, the inclusion of emotional and cognitive dimensions to multisensory virtual perception is what will bring the concept of ultra-reality. The aim of this paper is to show the work on a virtual acoustic system, which combines graphical and acoustic rendering (using a graphical engine and an audio engine). This system was implemented to render binaural auralizations and has been used for the rendering of a 16th century Valencian theatre, the so-called 'Corral de l'Olivera', which was rebuilt several times until the 18th century. It has been modeled graphically and for acoustic simulation. In addition, the auralization of diverse theatrical extracts of diverse Spanish authors was carried out. The consequences of this work will be reflected in future contributions on immersive theatre with greater immersive and emotional sensation. Thus, we contribute to the immersive virtual recovery of theatres that are historical cultural heritage

Plataforma Web 2.0 para la Sincronización Distribuida de Contenidos Multimedia e Interacción Social

Shared media experiences between geographically distributed users are gaining momentum. Relevant examples are Social TV, synchronous e-learning and multi-player online games. This paper presents a first release of Wersync, an adaptive web-based platform that provides distributed media synchronization and social interaction (via shared navigation control commands and text chat channels) across remote users. By using Wersync, users can create or join on-going sessions for concurrently consuming the same media content with other remote users in a synchronized manner. Additionally, Wersync provides two social presence mechanisms to encourage the participation of external users in on-going sessions and two privacy mechanisms. Wersync has been developed by exclusively relying on standard web-based technologies, which ensures cross-network, cross-platform and cross-device support. The evaluation results and a link to a demo video prove the satisfactory performance of Wersync, and its functionalities, respectively

Impacto de Parámetros de QoS en Aspectos de QoE: Análisis desde el Punto de Vista de la Sincronización Multimedia

La sincronización multimedia ha sido un área de investigación clave desde los inicios de los sistemas multimedia. En este artículo se ofrecen una visión general y un análisis sobre el impacto de varios parámetros de QoS en diferentes aspectos de la QoE, desde el punto de vista de la sincronización multimedia. En primer lugar, se presentan los diferentes tipos de sincronización multimedia y su relevancia para garantizar una QoE satisfactoria. En segundo lugar, se muestra que la magnitud de los retardos y su variabilidad en las redes actuales es bastante superior a los umbrales permisibles por los usuarios en diferentes tipos y ejemplos de sincronización multimedia, reflejando así su necesidad. En tercer lugar, se describe el impacto del ancho de banda y la tasa de pérdidas sobre la sincronización multimedia. Por último, se argumenta la influencia del uso de diferentes alternativas para conseguir la sincronización multimedia sobre varios factores de QoS y de QoE